FEW-SHOT SETTINGS

Improving and Simplifying Template-Based Named Entity Recognition (2023.eacl-srw)

Challenge:	Recent research shows that large language models pretrained using unsupervised approaches can achieve significant performance improvement on many downstream tasks.
Approach:	They propose an unsupervised approach to fine-tuning large language models using unsupervised approaches to many downstream tasks.
Outcome:	The proposed approach improves on four e-commerce applications and can achieve an average improvement of 10% in few-shot settings and 3.7% in data-rich settings over the standard approach.

Copied to clipboard

Murali Kondragunta, Olatz Perez-de-Viñaspre, Maite Oronoz

FS-DAG: Few Shot Domain Adapting Graph Networks for Visually Rich Document Understanding (2025.coling-industry)

Challenge:	Named Entity Recognition (NER) is traditionally approached as a sequence labeling task where a tag is predicted for each token.
Approach:	They propose to convert a Named Entity Recognition task into a seq2seq task by generating synthetic sentences using templates.
Outcome:	The proposed model outperforms the current state-of-the-art approach in resource-rich, low resource and domain transfer settings and the negative examples play an important role in its performance.

Copied to clipboard

Amit Agarwal, Srikant Panda, Kulbhushan Pachauri

Multimodality

Challenge:	Recent advances in vision-language models have significantly enhanced performance across various natural language processing and computer vision tasks.
Approach:	They propose a few shot domain adapting graph (FS-DAG) that leverages domain-specific and language/vision specific backbones within a modular framework to adapt to diverse document types with minimal data.
Outcome:	The proposed model is highly performant with less than 90M parameters, making it well-suited for complex real-world applications for information extraction tasks where computational resources are limited.

FPT: Feature Prompt Tuning for Few-shot Readability Assessment (2024.naacl-long)

Copied to clipboard

Ziyang Wang, Sanwoo Lee, Hsiu-Yuan Huang, Yunfang Wu

Few-Shot Text Generation with Natural Language Instructions (2021.emnlp-main)

Challenge:	Prompt-based methods lack crucial linguistic knowledge for readability assessment tasks such as word length, sentence length, and usage of different difficulty-level words.
Approach:	They propose a new prompt-based tuning framework that incorporates linguistic knowledge and a loss function to calibrate the similarity ranking order between categories.
Outcome:	The proposed framework outperforms the large language model gpt-3.5-turbo-16k in most cases.

Copied to clipboard

Timo Schick, Hinrich Schütze

Information Extraction & Text Mining

Challenge:	Existing approaches to text generation combine task descriptions and examples with supervised learning.
Approach:	They propose a method for text generation that is based on pattern-exploiting training.
Outcome:	The proposed approach improves on several summarization and headline generation datasets.

Building Adaptive Acceptability Classifiers for Neural NLG (2021.emnlp-main)

Copied to clipboard

Soumya Batra, Shashank Jain, Peyman Heidari, Ankit Arun, Catharine Youngs, Xintong Li, Pinar Donmez, Shawn Mei, Shiunzu Kuo, Vikas Bhardwaj, Anuj Kumar, Michael White

Information Retrieval

Challenge:	Existing approaches to generate synthetic data using simple sentence transformations and/or model-based techniques may not generate realistic error samples with respect to the NLG models.
Approach:	They propose a framework to train models to classify acceptability of responses generated by natural language generation models using a 2-stage approach . they use existing sentence transformations to generate samples that better resemble the output of the generation models.
Outcome:	The proposed approach outperforms existing techniques and can be used in few-shot settings using self-training.

A Generative Language Model for Few-shot Aspect-Based Sentiment Analysis (2022.findings-naacl)

Copied to clipboard

Ehsan Hosseini-Asl, Wenhao Liu, Caiming Xiong

Impact of Pretraining Term Frequencies on Few-Shot Numerical Reasoning (2022.findings-emnlp)

Challenge:	Pre-trained language models are often used to achieve state-of-the-art results . eval paper shows that generative language model can handle joint and multi-task settings .
Approach:	They propose to reformulate extraction and prediction tasks into a sequence generation task . they propose a generative language model with unidirectional attention that learns to accomplish the tasks via language generation .
Outcome:	The proposed model outperforms the state-of-the-art in few-shot and full-shot settings.

Copied to clipboard

Yasaman Razeghi, Robert L Logan IV, Matt Gardner, Sameer Singh

Few-shot Intent Classification and Slot Filling with Retrieved Examples (2021.naacl-main)

Challenge:	Pretrained language models have demonstrated ability to perform numerical reasoning by extrapolating from a few examples in few-shot settings.
Approach:	They investigate how well pretrained language models reason with terms less frequent in pretraining data.
Outcome:	The models are more accurate on instances whose terms are more prevalent, in some cases above 70% more accurate than the bottom 10%.

Copied to clipboard

Dian Yu, Luheng He, Yuan Zhang, Xinya Du, Panupong Pasupat, Qi Li

Information Retrieval

Challenge:	Existing methods for few-shot learning are based on labeled examples, but they are non-trivial . few-sshot learning is challenging due to the imbalance in the amount of data between the source and target domains.
Approach:	They propose retrieval-based methods for intent classification and slot filling tasks . they use a batch-softmax objective to learn similar contextualized representations for spans .
Outcome:	The proposed method outperforms previous systems on the CLINC and SNIPS benchmarks.

Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization (2023.findings-emnlp)

Copied to clipboard

Kaihang Pan, Juncheng Li, Hongye Song, Jun Lin, Xiaozhong Liu, Siliang Tang

DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization (2023.acl-long)

Challenge:	Existing methods for prompt tuning can overfit to few-shot training samples, causing overfitting . authors propose a new framework for prompt learning with supervised meta-learning .
Approach:	They propose a self-supervised meta-prompt learning framework with MEta-gradient Regularization for few-shot generalization that leverages self-recognized meta-learning with a diverse set of meta-tasks to learn a universal prompt initialization using only unlabeled data.
Outcome:	The proposed framework learns a universal prompt initialization for efficient adaptation using only unlabeled data.

Copied to clipboard

Yu Li, Baolin Peng, Pengcheng He, Michel Galley, Zhou Yu, Jianfeng Gao

Dialogue Systems & Conversational Agents

Challenge:	Existing methods for summarizing dialogues lack in taking into account the structure of dialogues and rely heavily on labeled data.
Approach:	They propose a pre-trained encoder-decoder model for summarizing dialogues in any new domain.
Outcome:	The proposed model outperforms existing methods on six datasets and shows ROUGE scores in zero-shot and few-shot settings.

Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning (2024.findings-eacl)

Copied to clipboard

Lifu Tu, Jin Qu, Semih Yavuz, Shafiq Joty, Wenhao Liu, Caiming Xiong, Yingbo Zhou

AugNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation (2021.acl-long)

Challenge:	Cross-lingual transfer of language models trained on high-resource languages such as English has been limited due to the high cost of obtaining non-English conversational data.
Approach:	They introduce a parallel and large-scale multilingual conversation dataset that is used for cross-lingual alignment pretraining by translating the English-only Schema-Guided Dialogue dataset into 105 other languages.
Outcome:	The proposed model performs well on slot-filling and intent classification tasks, and is able to perform well in other languages.

Copied to clipboard

Xinnuo Xu, Guoyin Wang, Young-Bum Kim, Sungjin Lee

Low-Resource Compositional Semantic Parsing with Concept Pretraining (2023.eacl-main)

Challenge:	Large-scale conversational systems typically generate unnatural, robotic responses using template-based approaches.
Approach:	They propose a data augmentation approach that combines a self-trained neural retrieval model with a few-shot learned NLU model to automatically create MR-to-Text data from open-domain texts.
Outcome:	The proposed approach outperforms the state-of-the-art methods on the FewshotWOZ data in both BLEU and Slot Error Rate.

Copied to clipboard

Subendhu Rongali, Mukund Sridhar, Haidar Khan, Konstantine Arkoudas, Wael Hamza, Andrew McCallum

Prompt Combines Paraphrase: Teaching Pre-trained Models to Understand Rare Biomedical Words (2022.coling-1)

Challenge:	Semantic parsing is a key role in voice assistants by mapping natural language to structured meaning representations.
Approach:	They propose an architecture to perform domain adaptation automatically with only a small amount of metadata about the new domain and without any new training data.
Outcome:	The proposed architecture outperforms existing models in low-resource settings.

Copied to clipboard

Haochun Wang, Chi Liu, Nuwa Xi, Sendong Zhao, Meizhi Ju, Shiwei Zhang, Ziheng Zhang, Yefeng Zheng, Bing Qin, Ting Liu

Self-training Strategies for Sentiment Analysis: An Empirical Study (2024.findings-eacl)

Challenge:	Pre-trained models perform poorly with limited data and rare biomedical words.
Approach:	They propose to use prompt to fine-tune pre-trained models for biomedical domain tuning with a simple approach.
Outcome:	The proposed method achieves up to 6% improvement in biomedical natural language inference task without any extra parameters or training steps using few-shot vanilla prompt settings.

Copied to clipboard

Haochen Liu, Sai Rallabandi, Yijing Wu, Parag Dakle, Preethi Raghavan

mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinations (2023.findings-emnlp)

Challenge:	Sentiment analysis is a crucial task in natural language processing.
Approach:	They propose to leverage a small amount of labeled and unlabeled data to train models with self-training.
Outcome:	The proposed method improves the performance of small language models in several few-shot settings while reducing the cost of annotations.

Copied to clipboard

Jonas Pfeiffer, Francesco Piccinno, Massimo Nicosia, Xinyi Wang, Machel Reid, Sebastian Ruder

Bi-Granularity Contrastive Learning for Post-Training in Few-Shot Scene (2021.findings-acl)

Challenge:	Recent large language models display surprising multilingual capabilities despite being pre-trained on English data.
Approach:	They propose a multilingual sequence-to-sequence model that disentangles language-specific information from language-agnostic information.
Outcome:	The proposed model outperforms existing models on representative natural language understanding and generation tasks in 40+ languages.

Copied to clipboard

Ruikun Luo, Guanhuan Huang, Xiaojun Quan

Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysis (2021.naacl-main)

Challenge:	Existing approaches to fine-tune pre-trained models to downstream tasks are limited by labeled examples.
Approach:	They propose to apply post-training on unlabeled task data before fine-tuning by contrastive learning that considers either token-level or sequence-level similarity.
Outcome:	Empirical results show that contrastive masked language modeling surpasses other methods in few-shot settings without the need for data augmentation.

Copied to clipboard

Xiao Liu, Da Yin, Yansong Feng, Yuting Wu, Dongyan Zhao

Structured Data in NLP

Challenge:	Existing studies focus on analyzing structured data, while mining causal relationship among factors from unstructured data is of great importance.
Approach:	They propose a graph-based causal inference framework which builds causal graphs from fact descriptions without much human involvement.
Outcome:	The proposed framework can capture nuance from fact descriptions among confusing charges and provide explainable discrimination in few-shot settings.

LiST: Lite Prompted Self-training Makes Parameter-efficient Few-shot Learners (2022.findings-naacl)

Copied to clipboard

Yaqing Wang, Subhabrata Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Awadallah, Jianfeng Gao

PromptFix: Few-shot Backdoor Removal via Adversarial Prompt Tuning (2024.naacl-long)

Challenge:	LiST is an efficient method for fine-tuning large pre-trained language models in few-shot learning settings.
Approach:	They propose a method for efficient fine-tuning of large pre-trained language models in few-shot settings using self-training and meta-learning.
Outcome:	The proposed method outperforms GPT-3 in-context learning by 33% on few-shot tasks.

Copied to clipboard

Tianrong Zhang, Zhaohan Xi, Ting Wang, Prasenjit Mitra, Jinghui Chen

Autonomous Aspect-Image Instruction a2II: Q-Former Guided Multimodal Sentiment Classification (2024.lrec-main)

Challenge:	Existing studies have shown that pre-trained language models can be backdoored such that model behavior is manipulated when trigger tokens are presented.
Approach:	They propose a backdoor mitigation strategy for NLP models via adversarial prompt-tuning in few-shot settings that uses two extra sets of soft tokens which approximate the trigger and counteract it respectively.
Outcome:	The proposed method keeps model parameters intact and only utilizes two extra sets of soft tokens which approximate the trigger and counteract it respectively.

Copied to clipboard

Junjia Feng, Mingqian Lin, Lin Shang, Xiaoying Gao

Information Retrieval

Challenge:	Existing methods to identify sentiment polarities of aspects are limited by the limited multimodal data available.
Approach:	They propose to use instruction tuning paradigm to combine language and vision data to combine text and image modalities.
Outcome:	The proposed model achieves state-of-the-art on benchmark datasets and in few-shot settings.

Eliciting and Understanding Cross-task Skills with Task-level Mixture-of-Experts (2022.findings-emnlp)

Copied to clipboard

Qinyuan Ye, Juan Zha, Xiang Ren

In-Context Learning for Few-Shot Dialogue State Tracking (2022.findings-emnlp)

Challenge:	Pre-trained transformer models are capable of multitasking on diverse NLP tasks, but little is known about how multitaskability and cross-task generalization is achieved.
Approach:	They propose to use a transformer-based mixture-of-expert model with a router component to choose among experts dynamically and flexibly.
Outcome:	The proposed models improve the average performance gain (ARG) metric by 2.6% when adapting to unseen tasks, and by 5.6% in zero-shot generalization settings.

Copied to clipboard

Yushi Hu, Chia-Hsuan Lee, Tianbao Xie, Tao Yu, Noah A. Smith, Mari Ostendorf

ASCM: An Answer Space Clustered Prompting Method without Answer Engineering (2022.findings-acl)

Challenge:	Existing methods for zero-shot and few-shot learning dialogue state tracking are hard and expensive.
Approach:	They propose an in-context learning framework for zero-shot and few-shot learning dialogue state tracking (DST) a large pretrained language model takes a test instance and a few exemplars as input and directly decodes the dialogue state .
Outcome:	The proposed framework outperforms state-of-the-art models in few-shot settings . it is flexible and scalable, and requires less data to adapt to new domains and scenarios .

Copied to clipboard

Zhen Wang, Yating Yang, Zhou Xi, Bo Ma, Lei Wang, Rui Dong, Azmat Anwar

In-context Learning for Few-shot Multimodal Named Entity Recognition (2023.findings-emnlp)

Challenge:	Pre-trained language models have shown a great impact on NLP tasks.
Approach:	They propose an answer space clustered prompting model and a synonym initialization method that automatically categorizes all answer tokens in a semantic-clustered embedding space.
Outcome:	Experiments show that the proposed method outperforms existing state-of-the-art methods in few-shot settings.

Copied to clipboard

Chenran Cai, Qianlong Wang, Bin Liang, Bing Qin, Min Yang, Kam-Fai Wong, Ruifeng Xu

Multimodality

Challenge:	Existing methods for named entity recognition are time-consuming and laborintensive.
Approach:	They propose a few-shot multimodal named entity recognition task that uses few examples to locate and identify named entities for a text-image pair.
Outcome:	The proposed framework outperforms baselines under several few-shot settings.

KECP: Knowledge Enhanced Contrastive Prompting for Few-shot Extractive Question Answering (2022.emnlp-main)

Copied to clipboard

Jianing Wang, Chengyu Wang, Minghui Qiu, Qiuhui Shi, Hongbin Wang, Jun Huang, Ming Gao

On-the-fly Definition Augmentation of LLMs for Biomedical NER (2024.naacl-long)

Challenge:	Extractive Question Answering (EQA) is one of the most essential tasks in Machine Reading Comprehension (MRC).
Approach:	They propose a framework that transforms extractive question answering into a non-autoregressive Masked Language Modeling (MLM) generation problem.
Outcome:	The proposed framework outperforms state-of-the-art approaches in few-shot learning scenarios by a large margin.

Copied to clipboard

Monica Munnangi, Sergey Feldman, Byron Wallace, Silvio Amir, Tom Hope, Aakanksha Naik

GNNavi: Navigating the Information Flow in Large Language Models by Graph Neural Network (2024.findings-acl)

Challenge:	Despite their general capabilities, LLMs struggle on biomedicalNER tasks due to specialized terminology and lack of training data.
Approach:	They propose a new knowledge augmentation approach which incorporates definitions of relevant concepts on-the-fly.
Outcome:	The proposed approach improves performance on biomedicalNER tasks by 15% (on average) The proposed method outperforms fine-tuned language models in few-shot settings.

Copied to clipboard

Shuzhou Yuan, Ercong Nie, Michael Färber, Helmut Schmid, Hinrich Schuetze

Example-Driven Intent Prediction with Observers (2021.naacl-main)

Challenge:	Large Language Models (LLMs) exhibit strong In-Context Learning (ICL) capabilities when prompts with demonstrations are used.
Approach:	They propose a prompt-based parameter-efficient fine-tuning approach that leverages insights into ICL’s information flow dynamics and hardwires the desired information flow into the GNN.
Outcome:	The proposed approach surpasses prompt-based fine-tuning methods in few-shot settings by updating just 0.2% to 0.5% of parameters.

Copied to clipboard

Shikib Mehri, Mihail Eric

Text Classification

Challenge:	Prior work has shown that BERT-like models attribute a significant amount of attention to the [CLS] token, which results in diluted representations.
Approach:	They propose two approaches to improve generalizability of dialog system intent classification models by using observers and example-driven training.
Outcome:	The proposed models achieve state-of-the-art on three intent prediction datasets in both the full data and few-shot settings.

Unifying Cross-Lingual Transfer across Scenarios of Resource Scarcity (2023.emnlp-main)

Copied to clipboard

Alan Ansell, Marinela Parović, Ivan Vulić, Anna Korhonen, Edoardo Ponti

Multilinguality

Challenge:	Existing approaches to deal with resource scarcity have not been developed to deal effectively with the problem.
Approach:	They propose to use a set of tools to harness data from one or more high-resource "source" languages to compensate for a shortage of data in low-resourced "target" languages.
Outcome:	The proposed technique can be easily adapted to unseen languages, extending the range of the proposed technique and translation-based transfer more broadly.

Improving the Sample Efficiency of Prompt Tuning with Domain Adaptation (2022.findings-emnlp)

Copied to clipboard

Xu Guo, Boyang Li, Han Yu

POE: Process of Elimination for Multiple Choice Reasoning (2023.emnlp-main)

Challenge:	Prompt tuning is a technique for adapting large-scale pretrained language models for downstream tasks.
Approach:	They propose to condition a frozen pretrained language model with soft prompts from data . they propose to use a domain adaptation technique to regularize the decision boundary .
Outcome:	The proposed method outperforms full-model tuning in data-scarce settings by a large margin.

Copied to clipboard

Chenkai Ma, Xinya Du

Improving Compositional Generalization with Self-Training for Data-to-Text Generation (2022.acl-long)

Challenge:	Current language models perform well on multiple choice reasoning tasks, but the options are not treated equally.
Approach:	They propose a two-step scoring method that scores options and masks them to make the final prediction from the remaining options.
Outcome:	The proposed method is especially performant on logical reasoning tasks.

Copied to clipboard

Sanket Vaibhav Mehta, Jinfeng Rao, Yi Tay, Mihir Kale, Ankur Parikh, Emma Strubell

Text Generation

Challenge:	Data-to-text generation focuses on generating fluent natural language responses from structured meaning representations (MRs).
Approach:	They propose a template-based input representation that greatly improves the model’s generalization capability.
Outcome:	The proposed model improves tree accuracy by 46%+ and reduces slot error rates by 73%+ over the strong baselines on SGD and Weather benchmarks.

Thinking about how to extract: Energizing LLMs’ emergence capabilities for document-level event argument extraction (2024.findings-acl)

Copied to clipboard

Kai Shuang, Zhouji Zhouji, Wang Qiwei, Jinyu Guo

Information Extraction & Text Mining

Challenge:	Existing models for document-level event argument extraction (D-EAE) lack key feature forgetting and cross-event argument confusion.
Approach:	They propose a document-level event argument extraction method based on guided summarization and reasoning that leverages the emergence capabilities of large language models to highlight key event information.
Outcome:	The proposed method outperforms baseline models by 1.3% F1 and 1.6% F1 on WIKIEVENTS and RAMS.

Compositional Task-Oriented Parsing as Abstractive Question Answering (2022.naacl-main)

Copied to clipboard

Wenting Zhao, Konstantine Arkoudas, Weiqi Sun, Claire Cardie

Natural Language Interfaces

Challenge:	Task-oriented parsing (TOP) aims to convert natural language into machine-readable representations of specific tasks, such as setting an alarm.
Approach:	They propose to reduce TOP to abstractive question answering by using canonical paraphrasing to generate linearized parse trees.
Outcome:	The proposed technique outperforms state-of-the-art methods in full-data settings while achieving dramatic improvements in few-shot settings.

Pretraining Language Models Using Translationese (2024.emnlp-main)

Copied to clipboard

Meet Doshi, Raj Dabre, Pushpak Bhattacharyya

Text Generation

Challenge:	a recent study shows that large language models perform well in low-resource languages . a vast majority of languages don't have comparable data as compared to English .
Approach:	They propose to use Translationese as synthetic data for pre-training language models for low-resource languages.
Outcome:	The proposed method reduces performance of LMs trained on clean data in Indian languages . the proposed model performs better in English than in other languages, but is not comparable to English.

CHEF in the Language Kitchen: A Generative Data Augmentation Leveraging Korean Morpheme Ingredients (2023.emnlp-main)

Copied to clipboard

Jaehyung Seo, Hyeonseok Moon, Jaewook Lee, Sugyeong Eo, Chanjun Park, Heuiseok Lim

Does Meta-learning Help mBERT for Few-shot Question Generation in a Cross-lingual Transfer Setting for Indic Languages? (2022.coling-1)

Challenge:	Korean morphological variations present unique opportunities and challenges in natural language processing (NLP), necessitating an advanced understanding of morpheme-based sentence construction.
Approach:	They propose a method to replicate morphological transformations inherent in Korean sentences based on lexical and functional morphemes through generative data augmentation.
Outcome:	The proposed method improves performance in Korean multiple classification datasets without incurring external data usage.

Copied to clipboard

Aniruddha Roy, Rupak Kumar Thakur, Isha Sharma, Ashim Gupta, Amrith Krishna, Sudeshna Sarkar, Pawan Goyal

Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation (2026.findings-acl)

Challenge:	Existing approaches to few-shot Question Generation (QG) are limited and require manual annotation.
Approach:	They propose to use multilingual BERT to perform few-shot question generation with cross-lingual transfer.
Outcome:	The proposed model improves in few-shot QG and human evaluation confirms it.

Copied to clipboard

Yuxiang Wang, Xinnan Dai, Wenqi Fan, Yao Ma

Mutual Reinforcement of LLM Dialogue Synthesis and Summarization Capabilities for Few-Shot Dialogue Summarization (2025.findings-naacl)

Challenge:	Recent studies focus on performance benchmarks without fully comparing LLMs to graph learning models.
Approach:	They evaluate off-the-shelf and instruction-tuned graph learning models across a variety of scenarios.
Outcome:	The proposed models outperform traditional graph learning models in few-shot settings, the authors show . their models out perform models with instruction tuning, and they show excellent generalization and robustness.

Copied to clipboard

Yen-Ju Lu, Ting-Yao Hu, Hema Swetha Koppula, Hadi Pouransari, Jen-Hao Rick Chang, Yin Xia, Xiang Kong, Qi Zhu, Xiaoming Simon Wang, Oncel Tuzel, Raviteja Vemulapalli

Natural Language Interfaces

Challenge:	Empirical results demonstrate that our method improves dialogue summarization, achieving a 1.5% increase in ROUGE scores and a 0.3% improvement in BERT scores in few-shot settings.
Approach:	They propose Mutual Reinforcing Data Synthesis (MRDS) within large language models to enhance few-shot dialogue summarization task.
Outcome:	Empirical results show that the proposed method improves dialogue summarization, achieving a 1.5% increase in ROUGE scores and a 0.3% improvement in BERT scores in few-shot settings.

Emergent Communication Pretraining for Few-Shot Machine Translation (2020.coling-main)

Copied to clipboard

Yaoyiran Li, Edoardo Maria Ponti, Ivan Vulić, Anna Korhonen

Template-free Prompt Tuning for Few-shot NER (2022.naacl-main)

Challenge:	state-of-the-art models that rely on multilingual pretrained encoders achieve sample efficiency in downstream applications, but lack abundant amounts of unlabelled text.
Approach:	They propose a method to pretrain neural networks via emergent communication from referential games by grounding communication on images as a crude approximation of real-world environments.
Outcome:	The proposed method significantly improves machine translation in few-shot settings and provides an evaluation protocol to probe the properties of emergent languages ex vitro.

Copied to clipboard

Ruotian Ma, Xin Zhou, Tao Gui, Yiding Tan, Linyang Li, Qi Zhang, Xuanjing Huang

Residual Prompt Tuning: improving prompt tuning with residual reparameterization (2023.findings-acl)

Challenge:	Prompt-based methods have been successfully applied in few-shot learning tasks . however, when applied to token-level labeling tasks, it would be time-consuming to enumerate the template queries over all potential entity spans.
Approach:	They propose a method to reformulate NER tasks as LM problems without templates.
Outcome:	The proposed method is 30.12 times faster than the template-based method under few-shot settings.

Copied to clipboard

Anastasiia Razdaibiedina, Yuning Mao, Madian Khabsa, Mike Lewis, Rui Hou, Jimmy Ba, Amjad Almahairi

AdaPrompt: Adaptive Model Training for Prompt-based NLP (2022.findings-emnlp)

Challenge:	Prompt tuning is one of the most parameter-efficient approaches for parameter-effective tuning of pre-trained language models.
Approach:	They propose to reparameterize soft prompt embeddings using a shallow network with a residual connection and use it to tune prompt embeds P.
Outcome:	The proposed method outperforms prompt tuning on SuperGLUE, T5-Base and BERT-Bass models and can reduce the prompt length by 10 times without hurting performance.

Copied to clipboard

Yulong Chen, Yang Liu, Li Dong, Shuohang Wang, Chenguang Zhu, Michael Zeng, Yue Zhang

STraTA: Self-Training with Task Augmentation for Better Few-shot Learning (2021.emnlp-main)

Challenge:	Prompt-based learning can tackle zero-shot and few-shot NLP tasks . authors propose a method that makes use of pre-trained language models .
Approach:	They propose to map NLP tasks into natural language prompts, which are then filled by pre-trained language models.
Outcome:	The proposed method outperforms standard prompt-based methods in few-shot settings.

Copied to clipboard

Tu Vu, Minh-Thang Luong, Quoc Le, Grady Simon, Mohit Iyyer

Revisiting the Practical Effectiveness of Constituency Parse Extraction from Pre-trained Language Models (2022.coling-1)

Challenge:	Recent advances in NLP demonstrate the effectiveness of applying large-scale pre-trained language models to downstream tasks.
Approach:	They propose a method that uses task augmentation to fine-tune unlabeled data.
Outcome:	The proposed approach improves sample efficiency across 12 few-shot benchmarks.

Copied to clipboard

Taeuk Kim

Enhancing Pre-Trained Generative Language Models with Question Attended Span Extraction on Machine Reading Comprehension (2024.emnlp-main)

Challenge:	Constituency Parse Extraction from Pre-trained Language Models (CPE-PLM) is a new paradigm that attempts to induce constituency parse trees based on the internal knowledge of pre-tried language models.
Approach:	They propose to use constituency parse trees from pre-trained language models to induce constituency trees by introducing a set of heterogeneous PLMs combined using two advanced ensemble methods.
Outcome:	The proposed approach is more effective than typical supervised parsers in few-shot settings.

Copied to clipboard

Lin Ai, Zheng Hui, Zizhou Liu, Julia Hirschberg

Bridging Text and Knowledge with Multi-Prototype Embedding for Few-Shot Relational Triple Extraction (2020.coling-main)

Challenge:	Extractive Machine Reading Comprehension (MRC) is a challenging field in the field of Natural Language Processing.
Approach:	They propose a Question-Attended Span Extraction module to address the limitations of generative approaches for extractive machine reading comprehension (MRC) . module significantly enhances performance of pre-trained generative language models, enabling them to surpass the extractive capabilities of advanced Large Language Models (LLMs)
Outcome:	The QASE module surpasses state-of-the-art models in few-shot settings.

Copied to clipboard

Haiyang Yu, Ningyu Zhang, Shumin Deng, Hongbin Ye, Wei Zhang, Huajun Chen

Exploring Hybrid Question Answering via Program-based Prompting (2024.acl-long)

Challenge:	Existing approaches to supervised relational triple extraction require huge amounts of labeled data.
Approach:	They propose a multi-prototype embedding network model to extract the composition of relational triples from unstructured text.
Outcome:	The proposed method improves the performance of the few-shot relational triple extraction problem.

Copied to clipboard

Qi Shi, Han Cui, Haofeng Wang, Qingfu Zhu, Wanxiang Che, Ting Liu

Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning (2023.acl-long)

Challenge:	Existing approaches to question answering over heterogeneous data are limited due to large scale of information and organic coupling of heterogenous data.
Approach:	They propose a program-based prompting framework for hybrid question answering tasks . it integrates various functions to perform hybrid information-seeking over data .
Outcome:	The proposed framework surpasses baseline systems and achieves the best performance under the fewshot settings.

Copied to clipboard

Tianxiang Sun, Zhengfu He, Qin Zhu, Xipeng Qiu, Xuanjing Huang

RISCORE: Enhancing In-Context Riddle Solving in Language Models through Context-Reconstructed Example Augmentation (2025.coling-main)

Challenge:	Prompt tuning is a parameter-efficient approach to adapting pre-trained language models to downstream tasks.
Approach:	They propose to combine pre-trained modules with pre-trains to boost prompt tuning for few-shot learning.
Outcome:	The proposed model outperforms prompt tuning, full model tuning, and prior prompt pre-training methods in few-shot learning settings.

Copied to clipboard

Ioannis Panagiotopoulos, George Filandrianos, Maria Lymperaiou, Giorgos Stamou

Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning (2024.emnlp-main)

Challenge:	Existing methods for prompting Large Language Models (LLMs) are lacking in advanced reasoning skills.
Approach:	They propose a method that generates and utilizes contextually reconstructed sentences to generate few-shot exemplars.
Outcome:	The proposed method significantly improves the performance of large language models in vertical and lateral thinking tasks, surpassing traditional exemplar selection strategies across a variety of few-shot settings.

Copied to clipboard

Shuai Zhao, Meihuizi Jia, Anh Tuan Luu, Fengjun Pan, Jinming Wen

Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models (2023.emnlp-main)

Challenge:	In-context learning has shown high efficacy in several NLP tasks, especially in few-shot settings.
Approach:	They propose a backdoor attack method that poisons demonstration examples and poisons the demonstration context, preserving the model's generality.
Outcome:	The proposed method can make models behave in alignment with predefined intentions without fine-tuning the model.

Copied to clipboard

Shuai Zhao, Jinming Wen, Anh Luu, Junbo Zhao, Jie Fu

DSPM-NLG: A Dual Supervised Pre-trained Model for Few-shot Natural Language Generation in Task-oriented Dialogue System (2023.findings-acl)

Challenge:	ProAttack is a novel and efficient method for performing clean-label backdoor attacks based on the prompt, which uses the prompt itself as a trigger.
Approach:	They propose a method for performing clean-label backdoor attacks based on the prompt, which uses the prompt itself as a trigger.
Outcome:	The proposed method achieves state-of-the-art performance on several NLP tasks, particularly in few-shot settings.

Copied to clipboard

Yufan Wang, Bowei Zou, Rui Fan, Ai Ti Aw, Tingting He

Few-Shot Named Entity Recognition: An Empirical Baseline Study (2021.emnlp-main)

Challenge:	Existing models for few-shot natural language generation are based on a dual correlation between NLG and SLU from the perspective of probability.
Approach:	They propose a dual supervised pre-trained model to regularize the pre-training process . they use a probabilistic approach to learn the dual correlation between NLG and SLU .
Outcome:	The proposed model outperforms the previous state-of-the-art models on a few-shot dataset.

Copied to clipboard

Jiaxin Huang, Chunyuan Li, Krishan Subudhi, Damien Jose, Shobana Balakrishnan, Weizhu Chen, Baolin Peng, Jianfeng Gao, Jiawei Han

Making Pre-trained Language Models Better Learn Few-Shot Spoken Language Understanding in More Practical Scenarios (2023.findings-acl)

Challenge:	Existing methods to build named entity recognition systems with limited labeled data are lacking.
Approach:	They propose three orthogonal schemes to build named entity recognition systems when labeled data is limited.
Outcome:	The proposed NER systems outperform existing methods on few-shot and training-free settings.

Copied to clipboard

Yufan Wang, Jie Mei, Bowei Zou, Rui Fan, Tingting He, Ai Ti Aw

KnowVrDU: A Unified Knowledge-aware Prompt-Tuning Framework for Visually-rich Document Understanding (2024.lrec-main)

Challenge:	Existing few-shot Spoken Language Understanding models need to be trained on a set of data-rich source domains and adapt to the target domain with a few examples.
Approach:	They propose a scenario where only a pre-trained language model and a few labeled examples are used to train few-shot SLU models.
Outcome:	The proposed model outperforms existing models on few-shot settings by reducing the number of slot labels and reducing training complexity.

Copied to clipboard

Yunqi Zhang, Yubo Chen, Jingzhe Zhu, Jinyu Xu, Shuai Yang, Zhaoliang Wu, Liang Huang, Yongfeng Huang, Shuai Chen

TrojanWave: Exploiting Prompt Learning for Stealthy Backdoor Attacks on Large Audio-Language Models (2025.emnlp-main)

Challenge:	Existing methods for integrating layout and image features into pre-training language models are not suitable for few-shot settings.
Approach:	They propose to reformulate VrDU tasks into a single question-answering format with task-specific prompts and train the pre-trained model with the parameter-efficient prompt tuning method.
Outcome:	The proposed framework can be used in few-shot settings and reduces data requirements.

Copied to clipboard

Asif Hanif, Maha Tufail Agro, Fahad Shamshad, Karthik Nandakumar

Representation Projection Invariance Mitigates Representation Collapse (2023.findings-emnlp)

Challenge:	Recent studies have shown that ALMs are vulnerable to adversarial attacks.
Approach:	They propose a backdoor attack tailored to the prompt-learning setting in frozen audio-language models.
Outcome:	The proposed method injects backdoors solely through learnable prompts, making it highly scalable and effective in few-shot settings.

Copied to clipboard

Anastasia Razdaibiedina, Ashish Khetan, Zohar Karnin, Daniel Khashabi, Vivek Madan

A Decoupled Multi-Agent Framework for Complex Text Style Transfer (2025.findings-emnlp)

Challenge:	Fine-tuning contextualized representations by pre-trained models can lead to representation degradation, which can result in instability, sub-optimal performance, and weak generalization.
Approach:	They propose a regularization method to maintain the information content of representations and reduce representation collapse during fine-tuning.
Outcome:	The proposed method outperforms baselines on most tasks and improves out-of-distribution performance.

Copied to clipboard

Lingxi Zhang, Yu-Neng Chuang, Guanchu Wang, Ruixiang Tang, Xuanting Cai, Rajesh Shenoy, Xia Hu

Text Generation

Challenge:	Existing models for text style transfer struggle with complex styles . existing models perform well on simple styles like sentiment and formality .
Approach:	They propose a multi-agent self-check framework that includes a large language model as a planner for disentangling subtasks and expert agents for executing the subtask.
Outcome:	The proposed framework significantly improves style strength and content preservation on simple and complex style datasets.

How to Make Large Language Models Generate 100% Valid Molecules? (2025.emnlp-main)

Copied to clipboard

Wen Tao, Jing Tang, Alvin Chan, Bryan Hooi, Baolong Bi, Nanyun Peng, Yuansheng Liu, Yiwei Wang

Towards Few-shot Entity Recognition in Document Images: A Graph Neural Network Approach Robust to Image Manipulation (2024.lrec-main)

Challenge:	Large language models (LLMs) can learn to perform a wide range of tasks, but generating valid molecules using representations like SMILES is challenging in few-shot settings.
Approach:	They propose a language framework that converts invalid SMILES to SELFIES and LLMs as post-hoc correctors to ensure that the molecules generated by LLM are 100% valid.
Outcome:	The proposed model performs worse with SELFIES than with SMILES and improves on other metrics.

Copied to clipboard

Prashant Krishnan, Zilong Wang, Yangkun Wang, Jingbo Shang

Multimodality

Challenge:	Existing methods for named entity recognition from document images are limited in few-shot settings.
Approach:	They propose a framework which leverages the topological adjacency relationship among tokens by learning layout information with graph neural networks.
Outcome:	The proposed framework outperforms baselines under different few-shot settings and shows better performance to image manipulations.

Many-Shot Scaling of In-Context Learning with Self-Generated Demonstrations (2026.findings-acl)

Copied to clipboard

Zhengyao Gu, Henry Peng Zou, Yankai Chen, Aiwei Liu, Weizhi Zhang, Philip S. Yu

MTA:A Merge-then-Adapt Framework for Personalized Large Language Models (2026.acl-long)

Challenge:	In-context learning methods that use self-generated annotations do not scale to many-shot scenarios.
Approach:	They propose a framework analogous to semi-supervised learning that uses self-generated annotations instead of ground truth labels.
Outcome:	The proposed framework outperforms ground truth ICL under zero-shot, few-shot and many-shot settings.

Copied to clipboard

Xiaopeng Li, Yuanjin Zheng, Wanyu Wang, Wenlin Zhang, Pengyue Jia, Yingyi Zhang, Haiying He, Mengyang Ma, Yiqi Wang, Maolin Wang, Xuetao Wei, Xiangyu Zhao